Audiovisual speech enhancement based on the association between speech envelope and video features
نویسنده
چکیده
The low level acoustico-visual association reported by Yehia et al. (Speech Comm., 26(1):23-43, 1998) is exploited for audio-visual speech enhancement with natural video sequences. The aim of this study is to demonstrate that the redundant components of AV speech are extractible with a suitable representation which does not involve any categorization process. A comparative study is achieved between different types of audio features, including the initial Line Spectral Pairs (LSP) and 4-subbands envelope energy. A gain measure of the enhancement is applied for the comparison. The results clearly show that the coarse envelope features allows a better gain than the LSP.
منابع مشابه
A phonetically neutral model of the low-level audiovisual interaction
The improvement of detectability by visible speech cues found by Grant and Seitz (JASA, 108:1197-1208, 2000) has been related to the degree of correlation between acoustic envelopes and visible movements. This suggests that the audio and visual signals could interact early during the audio-visual perceptual process on the basis of audio envelope cues. On the other hand, acoustic-visual correlat...
متن کاملSpeech Enhancement using Adaptive Data-Based Dictionary Learning
In this paper, a speech enhancement method based on sparse representation of data frames has been presented. Speech enhancement is one of the most applicable areas in different signal processing fields. The objective of a speech enhancement system is improvement of either intelligibility or quality of the speech signals. This process is carried out using the speech signal processing techniques ...
متن کاملA New Method for Speech Enhancement Based on Incoherent Model Learning in Wavelet Transform Domain
Quality of speech signal significantly reduces in the presence of environmental noise signals and leads to the imperfect performance of hearing aid devices, automatic speech recognition systems, and mobile phones. In this paper, the single channel speech enhancement of the corrupted signals by the additive noise signals is considered. A dictionary-based algorithm is proposed to train the speech...
متن کاملSpeech enhancement based on hidden Markov model using sparse code shrinkage
This paper presents a new hidden Markov model-based (HMM-based) speech enhancement framework based on the independent component analysis (ICA). We propose analytical procedures for training clean speech and noise models by the Baum re-estimation algorithm and present a Maximum a posterior (MAP) estimator based on Laplace-Gaussian (for clean speech and noise respectively) combination in the HMM ...
متن کاملDesign and realisation of an audiovisual speech activity detector
For many speech telecommunication technologies a robust speech activity detector is important. An audio-only speech detector will give false positi-ves when the interfering signal is speech or has speech characteristics. The modality video is suitable to solve this problem. In this report the approach to and implementation of a decision-based audiovisual speech detector is given. Acoustic and v...
متن کامل